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OPTICAL MONITORING APPARATUS WITH 
IMAGE-BASED DISTANCE ACCOMMODATION 



Technical Field 

[0001] The present Invenilon pertains to an optical monitoring system that 
automatically monitors moving bodies (e.g., intruders, etc.)- 
Background and Summary 

[0002] Conventional and known optical monitoring systems for buildings, 
stores, etc. have video cameras (e.g,, security cameras) that form 
and transmit images to a security center* which is sometimes located 
at a remote location. In this type of monitoring system, human 
observers in the security center constantly view and monitor the 
transmitted monitor images to detect any monitored subject. In a 
security application, for example, the monitored subject could be a 
human intruder. A problem with such systems is that they require a 
large staff of human observ/ers. 

[0003] Also known are automated optical monitoring systems that use 
infrared sensors, etc. to detect infrared energy generated by or 
emitted from the monitored subjects (e.g., intruders). These 
automated monitoring devices minimize the personnel burden, but 
tliey can also detect small animals, such as rats, thereby having the 
problem of easily generating false alarms. 

[0004] Furthermore, optical monitoring systems have been considered 
for automatically identifying intruders with image recognition to detect 
a monitored subject in a monitored region. However, the size of an 
intruder on the image plane varies greatly according to the imaging 
distance. Hence, these systems suffer from the problem that 
intruders cannot be uniformly image-recognized. 

[0005] Therefore, an object of the present invention is to provide an 

optical monitoring system capable of automatically identifying whether 
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a moving body is the monitored subject based upon information in the 
imaged image, 

[0006] In one embodiment, the invention includes a moving body 

detection sub-system that images a monitored region and detects a 
moving body from changes over time in the Image of the monitored 
region. A speed detection sub-system detects the speed of a moving 
body in the image plane (i.e., speed of the moving body image in the 
image plane), A scale detection sub-system detects the size of a 
moving body in the image plane (I.e., size of the moving body in the 
imagis plane). A moving body estimation sub-system decides 
whether a moving body is the monitored subject (e.g., a human being) 
based on the image plana speed detected by the speed detection 
sub-system and the image plane size detected by the scale detection 
sub-system. 

[0007] The "image plane speed'' and "image plane size" referred to 

above are relative parameters that change together depending on the 
imaging distance to the moving body. That is, when the imaging 
distance is large, the image plane size of the moving body is small. 
When this happens the moving body's image plane speed is also a 
small percentage, like the reduction percentage of the image plane 
size. 

[0008] Therefore it is possible to easily cancel the effect of imaging 

distance included in both parameters by performing processing to find 
the ratio between image plane speed and image plane size, for 
example. By reducing the effect of, or accommodating, imaging 
distance based upon image information in this way, the moving body 
estimation sub-system of the present invention can identify a moving 
body without being affected by the variations in image plane size due 
to imaging distance. 

[0009] In another embodiment, the moving body estimation sub-system 
has an actual scale estimation sub-system that estimates the actual 
size of a moving body based on the image plane speed detected by 
the speed detection sub-system and the image plane size detected by 
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the scale detection sub-system. The moving body estimation sub- 
system decides vi/hether a moving body is the monitored subject 
based on the estimated moving body's actual size. 

[001 OJ In this embodiment, the actual scale estimation sub-system uses 
an image plane ruler (i.e., length standard) for movement speed of 
the moving body and converts the image plane size of the moving 
body into an actual size that does not depend on the imaging 
distance. The moving body estimation sub-system decides whether a 
moving body is the monitored subject based on this actual size, and is 
essentially unaffected by imaging distance. 

[001 1] In another embodiment, the moving body estimation sub-system 
has a correlation relationship storage sub-system that stores the 
correlation relationships between the image plane speed and image 
plane size of assumed moving bodies. In addition, the correlation 
relationship storage sub-system has a class estimation sub-system 
that checks the image plane speed detected by the speed detection 
sub-system and the image plane size detected by the scale detection 
sub-system against the correlation relationships stored in the 
correlation relationship storage sub-system. The correlation 
relationship storage sub-system then estimates the class of the 
moving body and decides whether the moving body is the monitored 
subject based on the estimated class of the moving body, 

[0012] For example, if the "image plane speed" to "image plane size" 

correlation relationship of a nimble animal such as a cockroach or bee 
were applied as-is to the height of a human being, that movement 
speed would be a speed that far surpassed the world record in the 
sprint (about 2 seconds for a 100-meter run). This Illustrates that the 
"image plane speed" to 'Image plane size" correlation relationship 
varies according to the class of animal, 

[0013] In this embodiment, the correlation relationship storage sub- 
system stores 'Image plane speed" to "image piane size" correlation 
relationships for assumed moving body classes. The class 
estimation sub-system estimates whether a moving body belongs to 
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an assumed animal class by checking the image plane speed and 
image plane size found from the moving body image against this 
correlation relationship. With this sort of estimation operation it 
possible to Identify a moving body with almost no effect from 
differences in image plane size due to Imaging distance, 

[0014] In another embodiment, the moving body estimation sub-system 

has a moving body evaluation sub-system that calculates an 
evaluation value indicating the certainty that the moving body is the 
monitored subject. The evaluation value is based on the image plane 
speed detected by the speed detection sub-system and the image 
plane size detected by the scale detection sub-system. The moving 
body estimation sub-system decides whether the moving body is the 
monitored subject based on the evaluation value of the moving body 
evaluation sub-systam. 

[001 5] The "image plane speed" to ''image plane size" correlation 

relationship varies according to the class of animal. Therefore it is 
possible to calculate an evaluation value indicating the certainty of 
being the monitored subject by evaluating the extent to which these 
two parameters match the correlation relationship of the monitored 
subject. Therefore, in this embodiment, a moving body is identified as 
the monitored subject or not based on the calculated evaluation 
value. This embodiment makes it possible to identify a moving body 
with almost no effect from differences In image plane size due to 
imaging distance, 

[001 6] In another embodiment, a moving body detection sub-system 
images a monitored region and detects a moving body from changes 
over time in the monitored region. A position detection sub-system 
detects the position of the moving body in the image plane, A scale 
detection sub-system detects the size of the moving body in the 
image plane. A moving body estimation sub-system decides whether 
the moving body is the monitored subject based on the image plane 
position (i.e., position of the moving body image in the image plane) 
detected by the position detection sub-system and the image plane 
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size (I.e., size of the moving body image in the image plane) detected 
by the scale detection sub-system, 

[0017] (n general, the image plane position found in this manner 

exiiibits specific tendencies according to the class of animal. For 
example, if it Is a human being there is a high possibility it will be 
positioned on a path in the screen. If it is a cockroach, it is not limited 
to a path; it may also be positioned on a wall surface. Therefore if the 
image plane position is on a wall surface, the moving body can be 
estimated to be a cockroach or the Wke, not a human being. Also, 
even if a moving body is positioned on a path, a human being and a 
cockroach clearly differ with regard to image plane size. Therefore it 
is possible to focus in to some extent on the class of animal and make 
an appropriate moving body identification based on two pieces of 
information — image plane position and image plane size. 

[001 8] As an alternative in the immediately preceding embodiment, the 
moving body estimation sub-system may have an actual scale 
estimation sub-system that estimates the actual size of the moving 
body based on the image plane position detected by the position 
detection sub-system and the image plane size detected by the scale 
detection sub-system. In this alternative^ the moving body estimation 
sub-system decides whether the moving body is the monitored 
subject based on the estimated actual size of the moving body. 

[0019] In another altemative in the immediately preceding embodiment, 

the moving body estimation sub-system has a correlation relationship 
storage sub-system that stores the correlation relationships between 
the Image plane position and image plane size of assumed moving 
bodies. A class estimation sub-system checks the image plane 
position detected by the position detection sub-system and the image 
plane size detected by the scale detection sub-system against the 
correlation relationships stored in the correlation relationship storage 
sub-system. The class estimation sub-system estimates the class of 
the moving body and decides whether the moving body is the 
monitored subject based on the estimated class of the moving body. 
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[0020] When a monitored region is determined and an imaged region is 
observed for a long period of time, unique tendencies appear with 
regard to the image plane positions that are passed through and the 
Image plane size, and these tendencies can vary according to the 
class of moving body. Therefore this sort of tendency is found as a 
correlation relationship through statistical processing, etc, and stored 
in advance in the correlation relationship storage sub-system. 

[0021] The class estimation sub-system checks the image plane 

position and image plane size of the current moving body against the 
correlation relationships stored in the correlation relationship storage 
sub-system. The class estimation sub-system estimates the class of 
the moving body based on the extent of the comparison. Through 
this sort of operation the present invention makes It possible to 
identify a moving body with almost no effect from differences in image 
plane size due to imaging distance, 

[0022] In another alternative in the immediately preceding embodiment, 
the moving body estimation sub-system has a moving body 
evaluation sub-system that calculates an evaluation value indicating 
the certainty that the moving body is the monitored subject based on 
the image plane position detected by the position detection sub- 
system and the image plane size detected by the scale detection sub- 
system. The moving body estimation sub-system decides whether 
the moving body is the monitored subject based on the evaluation 
value of the moving body evaluation sub-system. 

[0023] The "image plane position" to "image plane size" correlation 
relationship exhibits specific tendencies according to the class of 
moving body. Therefore it is possible to calculate an evaluation value 
indicating the certainty of being the monitored subject by evaluating 
the extent to which the detected image plane position and image 
plane size match the correlation relationship of the monitored subject. 
In this alternative, a moving body is identified as the monitored 
subject based on the evaluation value calculated in this way. This 
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makes it possible to identify a moving body with almost no effect from 
differences in image plane size due to imaging distance. 
[0024] As an alternative in any of the preceding embodiments, the 

moving body estimation sub-system decides whether a moving body 
is the monitored subject for a limited specified area of the monitored 
region. 

[0025] In this alternative, a moving body is identified for limited 

specified sites in the monitored region. By limiting specified sites in 
this manner it is possible to appropriately increase the decision 
accuracy. In addition, the range subject to moving body identification 
is limited, so it is possible to reduce the amount of calculation 
processing needed for identifying a moving body. 

[0026] As a result, a monitoring system according to the present 

invention can automatically identify intruders or other moving things 
with high reliability. Also, a monitoring system according to can 
perform automatic identification using only image plane information, 
so separate distance-finding devices are not necessary, and the 
monitoring system structure can be simple and inexpensive. 

[0027] Additional objects and advantages of the present invention will 
be apparent from the detailed description of the preferred 
embodiment thereof, which proceeds with reference to the 
accompanying drawings. 

Brief Description of the Drawings 

[0028] Fig. 1 is a block diagram of an optical monitoring system 
according to the present invention. 

[0029] Fig, 2 is a flow diagram showing the overall operation of first- 
third embodiments of the monitoring system. 

[0030] Fig. 3 is graph of imaging distance versus image plane size for 
explaining moving body identification using only image plane size. 

[0031] Fig. 4 is a drawing showing one example of a moving body 
image. 

[0032] Fig. 5 is a flow diagram of a monitored subject decision routine 
in the first embodiment. 
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[0033J Fig. 6 is a flow diagram of a monitored subject decision routine 

In the second embodiment. 
[0034J Fig, 7 is a flow diagram of a monitored subject decision routine 

in tlie third embodiment. 
[0035] Fig. 8 IS a flow diagram showing fourth-sixth embodiments of the 

overall operation of the monitoring system- 
[003GJ Fig. 9 is a flow diagram showing the monitored subject decision 

routine In the fourth embodiment. 
[0037] Fig. 1 0 is a flow diagram showing the monitored subject decision 

routine in the fifth embodiment, 
[0038] Fig. 11 is a flow diagram showing the monitored subject decision 

routine in the sixth embodiment. 
[0039] Fig. 12 is an optical schematic diagram illustrating the basic 

principle of the evaluation calculation in the sixth embodiment. 
[0040] Fig, 13 is a schematic circuit diagram of a motion detection 

senson 

[0041] Fig. 14 is a schematic circuit diagram of a deviation detection 
circuit. 

[0042] Fig. 15 is a schematic circuit diagram of an image signal 
generating circuit, 

Detaired Description of Preferred Embodrment 
[0043] First Embodiment 

[0044] Fig. 1 is a block diagram of an optical monitoring device 1 1 
according to the present invention. Optical monitoring device 1 1 
generally includes an imaging sub-system 12, a monitored subject 
detection sub-system 13, and an alarm sub-system 14. 

[004S] Imaging sub-system 12 includes an imaging lens 15 for 

receiving light from a monitored area and imaging the light at an 
image plane of a solid-state imaging device 16 for motion detection. 
An exaniple of solid-state imaging device 16 is described in Japanese 
Laid-open Patent Application Hei 11-8805. Solid-state imaging 
device 16 for motion detection is driven according to a timing control 
signal output 17 from a sensor drive circuit 18. 
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[0046] The solid-state imaging device 1 6 for motion detection 

generates an image signal 19 and a moving body image signal 20 
simultaneously in parallel. The image signal 19 is directed via an 
image signal processing circuit 21 to an image recorder or monitor 
device 22. The moving body image signal 20 is directed via a buffer 
circuit 23 (FIFO, etc) to a bus 24 in the monitored subject detection 
sub-system 13. 

[0047] The monitored subject detection sub-system 13 includes a 

microcomputer 25 and a memory circuit 26, The microcomputer 25 
processes the moving body image signal 20 and detects a monitored 
subject in the moving body image, as described below in greater 
detail. The microcomputer 25 sends an alarm signal 27 to the alarm 

jO sub-system 14 in response to detection of the monitored subject. 

=1^ [0048] The alarm part 14 includes a relay circuit 28 and a 

III communication circuit 29. In response to an alarm signal 27 from the 

microcomputer 25, the relay circuit 28 outputs a control signal 30 to 
the switch image recorder 22 to recording status or light an alarm 

o lamp (not shown). Also, in response to an alarm signal from the 

jijj microcomputer 25, the communication circuit 29 sends an alarm 

\n signal 31 to a distant security center, etc. (not shown). 

u [0049] Overall Operation of this Embodiment 

[0050] Rg. 2 is a flow diagram showing the overall operation of the 

optical monitoring device 11, 

[0051] Step SI : First, the microcomputer 25 acquires the moving body 

image signal 20 via the memory circuit 23, and temporarily stores it in 

the -buffer circuit 26, _ , 

memory T. Sl^lw^a. 

[0052] Step S2: In order to limit the monitored region, the -^oo\ .S^24r 

microcomputer 25 masks the moving body image signal 20 in a 

predetermined image region. 

[0053] Step S3: The microcomputer 25 detects a region indicating a 

moving body from the masked moving body image signal 20. and 

calculates the image plane position (for example, the position of the 
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moving body centroid, the position of its vertical and horizontal edges, 
etc.) and image plane size of the moving body, 
[0054] Step S4: The microcomputer 25 attempts a first-stage decision 
by deciding whether or not the moving body is the monitored subject 
(e.g,, a human being) by deciding whether or not the image plane size 
is less than a threshold value. 
[0055] Fig. 3 Is a graph of imaging distance versus image plane size for 
explaining this first-stage decision. The horizontal axis in Fig. 3 
shows imaging distance from the Imaging part 12 to the moving body. 
The vertical axis shows the image plane size of a human being, small 
animal, and insect in pixel units at the image plane. This graph 
assumes that the height of a human being is about 1 .6 m, the body 
5 length of a small animal is 0.3 m, and the body length of an insect Is 

about 0.1 m, 

in [0056] In the sort of case shown in Fig. 3, by making the threshold 

value the minimum image plane size at which a human being appears 
! =^ (here it Is 80 pixels) and making a first-stage decision, it Is possible to 

1^ reliably exclude from the moving body image as non-human "a small 

jfl animal more than 2 m away" and "an insect more than 0,7 m away," 

in Therefore when the Image plane size of the moving body is less than 

j3 the threshold value of 80 pixels, the microcomputer 25 quickly 

decides that "the moving body in the monitored region is not a human 
being" and returns the operation to step Si . 
[0057] On the other hand, when the image plane size of the moving 
body equals or exceeds the threshold value of 80 pixels, the 
microcomputer 25 decides that "the moving body in the monitored 
region may be a human being" and shifts the operation to step S5 in 
order to make a more detailed decision, 
[0058] Step S5: The microcomputer 25 decides whether or not the 

image plane position was recorded in the previous frame. If it was 
recorded, the microcomputer 25 shifts the operation to step S6. If 
image plane position of the previous frame was not saved, the 
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microcomputer 25 decides that this corresponds to the first frame 
when the moving body appears, and shifts the operation to step S10. 

[0059] Step S6: The microcomputer 25 finds the interval between the 
Image plane position in the previous frame and the image plane 
position in the current frame, and calculates the image plane speed. 
For example, for the moving body image shown in Fig. 4, the moving 
body moves exactly 12 pixels between one frame and another. If the 
imaging rate is 10 frames/second, for example, the moving body's 
movement speed is equivalent to 120 pixels/second. 

[0080] Step 87: Here the microcomputer 25 executes a monitored 

subject decision routine to be described later, and decides whether or 
not the moving body is the monitored subject based on the moving 
body's image plane size and Image plane speed. 

[0061] Step S8: If it decides the moving body is the monitored subject, 
the microcomputer 25 shifts the operation to step 89. On the other 
hand, if it decides the moving body is not the monitored subject, it 
shifts the operation to step SIC. 

[0062] Step S9: V^hen the monitored subject is detected the 

microcomputer 25 sends an alarm signal to the alarm part 14. 
Subsequently the microcomputer 25 shifts the operation to step S10, 

[0063] Step SI 0: For the next and subsequent detections of image 
plane speed the microcomputer 25 saves the moving body's image 
plane position in the current frame, Subsequently, the microcomputer 
25 returns the operation to step SI , 

[0064] Through the series of operations described above, the 
monitoring device can perform automated monitoring. 

[0065] Detailed Explanation of the Monitored Subject Decision Routine 

[0066] Fig. 5 is a How diagram showing the monitored subject decision 
routine of the first embodiment. The previously noted step S7 
decision routine will be explained in detail using Fig. 5. 

[0067] Step 81 1 : The microcomputer 25 assumes that the moving 
body's image plane speed is in the speed range assumed for the 
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monitored subject (in this case, a human being), and estimates or 
accommodates the moving body's imaging distance, 

[0068] For example, in the case of the moving body image shown in 
Fig. 4, the moving bod/s image plane speed is 120 pixels/frame, 
based upon the image plane speed of 12 pixels/frame and an 
exemplary image rate of 10 frames/second. For example, the optical 
system of imaging part 12 may project a subject with dimensions 1 m 
located at an imiaging of 5 m on the image plane as 100 pixels. 
Therefore, if we assume that the assumed human speed range of 0.8- 
1 .6 m/second is projected at an image plane speed of 120 
pixels/second, the imaging distance is 3.3-6.7 m. 

[0069] Step S12: The microcomputer 25 estimates the moving body's 
actual size from the estimated moving body's imaging distance and 
the moving body's image plane size, 

[0070] In the case of the moving body image shown in Fig. 4, the 

moving body's image plane size is 160 pixels. The estimated imaging 
distance is 3,3-6.7 m, so the actual size of the moving body is 
estimated to be 1.1-2.1 m. 

[0071] Step SI 3: Based on the moving body's actual size, the 

microcomputer 25 decides whether or not the moving body is the 
monitored subject (here, a human being). 

[0072] In the case of the moving body image shown in Fig. 4, the 

moving body's actual size is 1.1-2.1 m, which overlaps the size range 
assumed as human. Also, it clearly differs from the size assumed for 
a small animal or insect. Therefore the moving body shown in Fig. 4 
is identified as a human being. 

[0073] Incidentally, if the moving body shown in Fig, 4 is assumed to be 

a small animal with movement speed about 0.5-1.0 m/sec, the 
resulting estimated body length of the assumed small animal is 0,67 - 
1 .33 m. This is clearly different from the body length assumed from a 
rat or other small animal, and produces the decision "the moving body 
shown in Fig. 4 is not a small animal." 
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[00741 In addition, if the moving body shown in Fig. 4 is assumed to be 
an inject with movennent speed of 0,3-1 .0 m/sec, the resulting 
estimated body length of the assumed insect - 0,40-1 .33 m. This is 
clearly differeni from the body length assumed from a flying insect or 
other insect, and produces the decision "the moving body shown in 
Fig. 4 Is not an insect." In this manner the present invention makes it 
possible to Identify a moving body with almost no effect from 
differences in image plane size due to imaging distance, 

[0075] Effect of the First Embodiment 

[0076] Through the operations described above the first embodiment 
makes it possible to appropriately decide whether or not a moving 
body is the monitored subject regardless of the size of the moving 
body in the image plane. In particular, the first embodiment estimates 
the actual size of a moving body^ so It is suitable for monitoring 
applications where estimating actual scale is important, such as 
identifying whether a moving body is a child or an adult, 

[0077] Second Embodiment 

[0078] The constitution and overall operation of the second 

embodiment are the same as for the first embodiment (Fig, 1, Fig, 2). 
The operational feature of the second embodiment is the point about 
deciding whether a moving body is the monitored subject according to 
the decision routine shown in Fig. 6, 

[0079] Step S21 ; A "correlation relationship between image plane 
speed and image plane size*' statistically obtained from previous 
imaging test results is stored in the memory circuit 26 for each class 
of moving body. The microcomputer 25 estimates the moving body 
class by checking the moving body's image plane speed and image 
plane size against these correlation relationships, 

[0080] Step S22: The microcomputer 25 decides if the moving body is 
the monitored subject based on the estimated moving body class. 

[0081] Through the decision operation described above the second 

embodiment makes it possible to appropriately decide whether or not 
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a moving body is the monitored subject regardless of the size of the 
moving body in the image plane. 
[0082] Third Embodiment 

[0083] The constitution and overall operation of the third embodiment 
are the same as for the first embodiment (Fig. 1 , Fig, 2). The 
operational feature of the third embodiment is the point about 
deciding whether or not a moving body is the monitored subject 
according to the decision routine shown in Fig. 7, 

[0084] Step S31 : The microcomputer 25 calculates an evaluation value 
V indicating the certainty that a moving body is the monitored subject 
(e.g., a human being) according to the following equation: 

[0085] V = HA+WB-S, 

[0086] where H is the moving body's image plane body height, A is an 
image plane height evaluation coefficient, W is the moving body's 
image plane width, B is an image plane width evaluation coefficient, 
and S is image plane speed, 

[0087] Since a human being is a tall moving body, the image plane 
height evaluation coefficient A is set large in order to mal<e the 
evaluation value high when the moving body is a human being. Also, 
for human beings it should be noted that the image plane speed is 
relatively slow compared to the image plane size, so a negative 
evaluation coefficient is applied to the image plane speed. As a 
result, the evaluation value V calculated from the equation above 
gives a high value for a human being and gives a low value for an 
insect or small animal. 

[0088] Step S32: The microcomputer 25 compares the threshold value 
to the evaluation value, and decides whether or not the moving 
body is the monitored subject (e.g., a human being). 

[0089] Through the decision operation described above the third 

embodiment makes it possible to appropriately decide whether or not 
a moving body is the monitored subject regardless of the size of the 
moving body in the image plane. 
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[0090] Fourth Embodiment 

[0091] Fig. 8 is a flow diagram showing the overall operation of the 
fourth embodiment. As shown in Fig. 8, the overall operation of the 
fourth embodiment omits the operations related to image plane speed 
in the first embodiment (Fig, 2) (S5, S6, SIC). OthenA/ise, the 
constitution of the fourth embodiment is the same as for the first 
embodiment (Fig. 1). The operational feature of the fourth 
embodiment is the decision routine (Si 00) shown in Fig, 8, Fig, 9 is a 
flow diagram showing the specific operation of this decision routine. 

[0092] Step 841 : The association relationship between image plane 
position and imaging distance is stored in advance In the memory 
circuit 26. The microcomputer 25 estimates the moving bod/s 
Imaging distance by fitting the moving body's image plane position to 
this association relationship. 

[0093] Step S42: The microcomputer 25 estimates the moving body's 
actual size from the estimated moving bod/s Imaging distance and 
the moving body's image plane size, 

[0094] Step S43: The microcomputer 25 decides whether the moving 

body IS the monitored subject (e.g., a human being) based on the 
estimated moving body's actual size. 

[0095] Through the operation described above the fourth embodiment 
makes it possible to appropriately decide whether or not a moving 
body is the monitored subject regardless of the size of the moving 
body in the image plane. 

[0096] Fifth Embodiment 

[0097] The constitution of the fifth embodiment is the same as for the 
first embodiment (Fig, 1), and the overall operation of the fifth 
embodiment is the same as for the fourth embodiment (Fig. 8). The 
operational feature of the fifth embodiment is the point about deciding 
whether or not a moving body is the monitored subject according to 
the decision routine shown in Fig. 10. 

[0098] Step S51: A "correlation relationship between image plane 
position and image plane size" statistically obtained from previous 
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imaging test results is stored in the memory circuit 26 for each clas$ 
of moving body. The microcomputer 25 estimates the moving body 
class by checking the moving body's image plane position and image 
plana size against these correlation relationships. 

[0099] Step S52: The microcomputer 25 decides if the moving body is 
the monitored subject based on the estimated moving body class. 

[001 00] Through the decision operation described above the fifth 

embodiment makes it possible to appropriately decide whether or not 
a moving body is the monitored subject regardless of the size of the 
moving body in the image plane. 

[001 01] Sixth Embodiment 

[001 02] The constitution of the sixth embodiment is the same as for the 
first embodiment (Fig. 1), and the overall operation of the sixth 
embodiment is the same as for the fourth embodiment (Fig. 8). The 
operational feature of the sixth embodiment Is the point about 
deciding whether or not a moving body is the monitored subject 
according to the decision routine shown in Fig. 11, 

[001 03] Step 861 ; The microcomputer 25 calculates an evaluation value 
W indicating the certainty that a moving body is a human being 
according to the following equation; 

[00104] W = Ip/ |Yvp - Yip|, 

[00105] where Ip is the image plane size, Yyp is the Y-coordinate of the 
vanishing point, and Y|p is the Y-coordinate of the image plane 
position. 

[001 OB] The denominator term in this equation corresponds to lengths 
LI and L2 in the image plane shown in Fig. 12, for example. These 
lengths Li and L2 are lengths that are the same length L in the field 
of view but have undergone their respective perspective conversions, 
thereby providing an estimate or substitute parameter for the imaging 
distance. Therefore it is possible to use lengths LI and L2 for each 
moving body's ruler (i.e,, length standard) and compare the moving 
body sizes in the image plane. In the equation above, the image 
plane size is divided by this sort of denominator term* As a result, the 
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evaluation value W is calculated in proportion to the actual size of the 
moving body. 

[00107J Furthermore, Fig. 12 shows a case in which the position of the 
bottom edge of the moving body image is used as the image plane 
position, but the present invention is not limited to this 
implementation. In general, any image plane position that does not 
overlap the vanishing point can be evaluated by the aforesaid 
equation, For example, it is possible to use a moving body's centroid, 
top edge, left edge, right edge, etc, as the image plane position. 

[00108] Step S62: The microcomputer 25 compares the threshold value 
to the evaluation value W, and decides whether or not the moving 
body is the monitored subject (e.g., a human being). When doing so, 
if the evaluation value W is less than the threshold value, the moving 
body is too small to be a human being, so it is identified as an insect 
or small animal. And if it is equal to or greater than the threshold 
value, the size is that assumed for a human being, so it is identified 
as a human being. Furthermore, this threshold value should be 
varied in a way that is linked to any angle of view setting (zoom 
adjustment) of the imaging lens 15. 

[00109] Through the decision operation described above the sixth 

embodiment mal<es it possible to appropriately decide whether or not 
a moving body is the monitored subject regardless of the size of the 
moving body in the image plane. 

[001 10] Supplementary Embodiment Matters 

[001 1 1 1 The embodiments described above explained cases in which 
the decision operation regarding the monitored subject was 
performed independently, but the present invention is not limited to 
such implementations. For example, it is possible to perform a more 
highly reliable decision operation by combining a number of decision 
operations or by combining separate decision operations. 

[001 12] Also, in the aforesaid embodiments the image plane size was 
treated as one-dimensional data In the decision operation regarding 
the monitored subject, but the present invention is not limited to this 

17 



05/22/2,001 16:45 5032437058 
1113-014 



IPSOLON 



PAGE 19 



implementation. For example, the image plane size may also be 
treated as two-dimensional data (vertical width, horizontal width). In 
this case it is possible to add moving body shape information to the 
decision operation, and a more highly precise decision operation is 
possible. For example, the area of the moving body can be found by 
counting the mo'ving body's number of pixels, etc., and this area can 
be treated as the image plane size. 

[001 13] Also, in the embodiments described above an alarm signal was 
output to the alarm part 1 4 simultaneously with identification of the 
monitored subject. This has the excellent advantage that the 
monitored subject is reported quickly and the recording operation or 
alarm operation can begin swiftly. Nevertheless, the present 
invention is not limited to this implementation. For example, the alarm 
signal can be output if identification of the monitored subject 
continues for exactly a specified number of times, or if It is identified 
frequently. This has the advantags of preventing erroneous 
identification caused by noise, etc. 

[00114] Furthermore, the embodiments described above explained 
cases in which a human being was the monitored subject, but the 
present invention is not limited to this implementation. For example, 
an insect or small animal or other creature may be the monitored 
subject. The monitored subject may also be smoke or fire or other 
phenomena. 

[001 15] Also, the embodiments described above generated a moving 
body image signal using the solid-state imaging device 16 for motion 
detection. This has the excellent advantage that absolutely no 
external circuits are needed to generate the moving body image 
signal, and the overall structure of the monitoring device can be 
simple and inexpensive. Nevertheless, the present invention is not 
limited to this implementation. Of course it is also possible features 
such as an imaging element and image memory and image 
processing circuit, etc., and generate the moving body image signal. 
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[0011 6] Also, in the aforesaid first through third embodiments the image 
plane speed was found from the moving body movement interval 
between frames. Nevertheless, the present invention is not limited to 
this implementation. Of course the image plane speed can also be 
found from the moving body image's edge width or edge area, etc. 

[001 17] Furthermore, in the aforesaid first through third embodiments 
the momentary image plane speed was found. This has the 
advantage that moving body rdentlficatfon using image plane speed 
can be done in real time or at high frequency. Nevertheless, the 
present invention is not limited to this implementation. For example, it 
is also possible to find the average value, maximum value, or 
minimum value of the image plane speed and make the aforesaid 
decision using this value. This has the advantage that a moving body 
that is changing speed can be identified by a constant standard. 

[001 18] Effect of the Invention 

[001 19] The present invention decides whether a moving body is a 
predetermined monitored subject (e.g., a human being) based on 
Image plane speed and image plane size. This reduces the effect of 
imaging distance based on image plane speed and image plane size, 
so it is possible to appropriately decide whether the moving body is 
the monitored subject with almost no affect from the size of the 
moving body in the image plane. 

[00120] In one embodiment, the invention uses an image plane ruler 
(i.e., length standard) for the moving body's movement speed and 
converts the moving body's image plane size into an estimated actual 
size that does nolt depend on the imaging distance. Therefore it is 
suitable for monitoring applications that need to estimate actual scale, 
such as identifying whether a moving body is a child or an adult, 

[00121] Also, the invention can estimate the moving body class by 

checking an 'Image plane speed" to "image plane size" correlation 
relationship. In this case it is possible to accurately find the 
correlation relationship using statistical methods, etc. Therefore it is 
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possible to decide whether or not a moving body is the monitored 
subject with statistically reliable precision, 

[00122] tn addition, the invention can perform an evaluation calculation 
based on image plane speed and image plane size. In this case it is 
possible to decide whether or not a moving body is the monitored 
subject with a relatively small amount of calculation processing. 

[00123] On the other hand, the invention can decide whether a moving 
body is the monitored subject based on image plane position and 
image plane size. In this case it is possible to more accurately decide 
whether a moving body is the monitored subject by adding 
information on its image plane position. In particular, the invention 
can convert the moving body's image plane size into an actual size 
that does not depend on imaging distance^ based on the image plane 
position. Therefore it is suitable for monitoring applications that need 
to estimate actual scale, such as identifying whether a moving body is 
a child or an adult. 

[00124] Also, the invention can estimate the moving body class by 

checking an "image plane position" to "image plane size" correlation 
relationship. In this case it is possible to accurately find the 
correlation relationship using statistical methods, etc. Therefore it is 
possible to decide whether or not a moving body is the monitored 
subject with statistically reliable precision. 

[00125] Also, the invention can perfonm an evaluation calculation based 
on image plane position and image plane size. In this case it is 
possible to decide whether or not a moving body is the monitored 
subject with a relatively small amount of calculation processing. 

[00126] Furthermore, the invention can decide whether a moving body is 
the monitored subject for a limited specified area of the monitored 
region. In this case it is possible to appropriately avoid a reduction in 
decision precision by the aforesaid invention due to special 
characteristics of the monitored region, and it is possible to increase 
the reliability of moving body Identification in the aforesaid invention. 
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Also, it is possible to reduce the amount of calculation in moving body 
identification. 

[00127] Due to the various effects of the present invention as described 
above, a monitoring device employing the present invention can 
automatically identify intruders or other moving things with high 
reliability. Also,, it performs automatic identification using only image 
plane information, so separately included distance-finding devices are 
not necessary, and the monitoring device structure can be made 
simple and inexpensive. 

[00128] Fig. 13 Is a schematic circuit diagram that shows the schematic 
structure of motion detection sensor 16 used in this optical monitoring 
apparatus. Moreover, motion detection sensor 16 shown in Fig. 13 is 
a solid-state camera element, and has several pixels 101 arranged in 
a matrix (in the figure, for simplicity, four pixels are shown arranged in 
a 2 X 2 matrix). 

[00129] Vertical reading lines 102a and 102b are installed on each 

column of pixels 101 arranged vertically. These are connected both to 
pixels 101 by way of transistor QX explained below, and to deviation 
detection circuit 103 and image signal generating circuit 106. 

[00130] The output of deviation detection circuit 103 is connected to shift 
register 104, and the output of image signal generating circuit 106 is 
connected to horizontal reading line 107 by way of horizontal reading 
switching transislors QH1 and QH2. 

[00131] Each of pixels 101 is comprised of photodiode PD that 

generates a charge corresponding to incident light, junction field 
effect transistor QA that outputs an electrical signal corresponding to 
the charge generated by photodiode PD, transmitting MOS transistor 
QT that transmits the charge generated by photodiode PD directly to 
the gate region of transistor QA» resetting transistor QP that 
discharges the charge accumulated in the gate region of transistor 
QA, and switching MOS transistor QX that connects or separates 
vertical reading lines 102a and 102b and transistor QA. 
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[00132] Fig. 1 4 is a schematic circuit diagram tliat shows the structure of 
deviation detection circuit 103, 

[00133] In this figure, deviation detection circuit 109 is comprised of 
switching MOS transistors QR and QS, capacitors GR and CS that 
accumulate a charge corresponding to the electrical signal outputted 
by pixel 101 at different timings, and comparison circuit XA that 
compares the charges accumulated in capacitors CR and CS, 

[00134] Fig. 15 is a schematic circuit diagram that shows the structure of 
image signal generating circuit 106. 

[00135] In this figure, image signal generating circuit 106 is comprised of 
capacitor CV that accumulates a charge corresponding to the 
electrical signal outputted by pixel 101, and transistor QV for applying 
sampling hold switching to capacitor CV, 

[00136] In motion detection sensor 16, the optical image obtained by 
lens 15 is imaged and subjected to photoelectric conversion by 
photodiode PD In each pixel 101 at a predetermined timing, 

[001 37] When the signal charge generated by photodiode PD by this 

type of photoelectric conversion is conducted to transistor QT in pixel 
101, It is transmitted to the gate of transistor QT. Following this, when 
transistor QT becomes nonconducting, the gate region of transistor 
QA becomes floating, but holds the above-mentioned signal charge 
by parasitic capacitance effect. That is, the gate region of transistor 
QA accumulates the signal charge generated by photodiode PD and 
acts as a memory that temporarily holds this, 

[00138] Next, the case is considered when the signal charge for the 

previous frame already is accumulated in the gate region of transistor 
QA and a new signal charge for the current frame is generated by 
photodiode PD. 

[00139] In this state, when transistor QX in pixel 101 and transistor QR 
in deviation detection circuit 103 become conducting, transistor QA 
acts as a source follower, and a charge corresponding to the signal 
charge for the previous frame accumulated in the gate region of 
transistor QA is charged to capacitor CR in deviation detection circuit 
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103 by way of vertical reading line 102, In addition, when transistor 
QP in pixel 101 becomes conducting, the signal charge accumulated 
in the gate region of transistor QA is discharged to initialize the 
transistor. 

[00140] Following this, when transistor QT in pixel 1 01 becomes 

conducting, the new signal charge for the current frame generated by 
photodiode PD is transmitted to the gate of transistor QA. In addition, 
when transistor QX in pixel 101 and transistor QS in deviation 
detection circuit 103 become conducting, transistor QA acts as a 
source follower, and a charge corresponding to the signal charge for 
the current frame accumulated in the gate region of transistor QA is 
charged to capacitor CS in deviation detection circuit 103 by way of 
vertical reading line 102. 

[00141] That Is, within deviation detection circuit 103, a charge 
corresponding to the signal charge for the previous frame is 
accumulated in capacitor CR and a charge corresponding to the 
signal charge for the current frame is accumulated in capacitor CS. 

[00142] In addition, the signal charge for the current frame transmitted to 
the gate of transistor QA in pixel 101 is held in the gate region of this 
transistor QA, and in the next frame, is used as the signal charge 
corresponding to the previous frame. 

[00143] Comparison circuit XA obtains the absolute value of the 

difference in signal voltage corresponding to the charges to capacitor 
GR and capacitor CS. In addition, comparison circuit XA outputs a 
signal indicating "V (or "0"; signal level that indicates that motion is 
present) when the absolute value obtained is greater than or equal to 
a set value, and outputs a signal indicating W (or "1": signal level that 
indicates that motion is not present) when the absolute value obtained 
is less than a set value. The signal outputted by comparison circuit 
XA in this way is outputted externally in sequence by way of shift 
register 104. 
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[00144] That is, a motion signal can be obtained easily by comparing 
signal voltages that correspond to signal charges corresponding to 
two continuous frames for each pixel. 

[00145] In addition, after transistor QP in pixel 1 01 becomes conducting, 
the signal charge accumulated in the gate region of transistor QA Is 
discharged, and the transistor is initialized, when transistor QX in 
pixef 101 and transistor QS in deviation detection circuit 103 become 
conducting, transistor QA acts as a source follower, and a charge 
corresponding to the initialized state of the gate region of this 
transistor QA is charged to capacitor CV in image signal generating 
circuit 106 by way of vertical reading line 102, In addition, the signal 
charged to capacitor CV in this way is held in capacitor CV even after 
transistor QV becomes nonconducting and capacitor CV becomes 
floating, 

[00146] Following this, when transistor QT in pixel 101 becomes 

conducting, the signal charge for the current frame generated by 
photodiode PD is transmitted to the gate region of transistor QA. In 
this state, when transistor QX in pixel 101 becomes conducting, 
transistor QA again acts as a source follower, and a signal 
corresponding to the signal charge accumulated in the gate region of 
transistor QA is inputted to capacitor CV in image signal generating 
circuit 106 by way of vertical reading line 102. 

[00147] In this case, because capacitor CV already holds a charge 
corresponding to the state after the gate region of transistor QA in 
pixel 101 was initialized, from the output of capacitor CV (the output 
of image signal generating circuit 106), a signal is outputted that 
corresponds to the difference between the signal determined by the 
state of accumulation of the signal charge for the current frame in the 
gate region of transistor QA in pixel 101 and the signal corresponding 
to the state after being initialized. 

[001 48] It is known that the signal corresponding to the state after the 
gate region of transistor QA in pixel 101 is Initialized contains voltage 
discrepancies between the gate and the source of transistor QA that 
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cause fixed pattern noise, and reset noise (so-called kTC noise) just 
after the gate region of transistor QA in pixel 101 is initialized that 
causes random noise. However, in this embodiment, image signal 
generating circuit 106 can obtain an image signai from which fixed 
pattern noise and random noise have been eliminated. 

[00149] Therefore, motion detection sensor 16 can output a motion 
signal and an image signal simultaneously, IVIoreover, the motion 
signal and Image signal outputted by motion detection sensor 16 in 
this way are supplied to processing circuit 21 and buffer circuit 23, 

[00150] In view of the many possible embodiments to which the 

principles of this invention may be applied, it should be recognized 
that the detailed embodiments are illustrative only and should not be 
taken as limiting the scope of the invention. Rather, I ciaim as my 
invention ail such embodiments as may come within the scope and 
spirit of the following claims and equivalents thereto. 
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